TITLE OF THE INVENTION 

RENDERING DEVICE 

BACKGROUND OF THE INVENTION 

Field of the Invention 
[0001] The present invention relates to rendering devices and, 
more specifically, to a rendering device which can be incorporated 
in a drive assistant device. In more detail, the rendering device 
generates a display image of around a vehicle based on an image 
captured by an image capture device fixedly placed in the vehicle . 

Description of the Background Art 
[0002] The drive assistant device incorporating such 
rendering device as above has been actively researched and 
developed. A conventional- type drive assistant device is 
mounted in a vehicle , and generally includes an image capture 
device, a rudder angle sensor, a computing unit, a rendering 
device, and a display device. The image capture device is fixedly 
placed in a predetermined position in the vehicle, and takes 
charge of capturing an image of an area defined by its own viewing 
angle. The resulting image is now referred to as a captured image. 
The rudder angle sensor is also fixed in a predetermined position 
in the vehicle, and detects to what degree the steering wheel of 
the vehicle is turned. Based on the detection result, the 
computing unit calculates a path estimated for the vehicle to take. 



The rendering device then renders the estimated path on the 
captured image, and the image generated thereby is such display- 
image as shown in FIG. 20. The display image is displayed on the 
display device. 

5 [0003] With such display image on the display device, a driver 
of the vehicle can know if his/her current steering will fit the 
vehicle in the parking space without colliding any obstacle in 
a close range. If his/her steering is not appropriate, the 
estimated path is displayed out of the parking space in the display 

Bj 10 image. Therefore, the driver can appropriately adjust the rudder 

O angle of the steering wheel. 

Ul [0004] There is another type of conventional drive assistant 

3 

2 device exemplarily disclosed in Japanese Patent examined 

y * 

Publication No. 2-36417 (1990-36417). The drive assistant 
- 15 device additionally carries an active sensor for measuring a 
distance between the vehicle and an obstacle observed near the 
estimated path. Based on the measurement result provided by the 
active sensor, the computing unit determines which part of the 
estimated path is to be rendered on the captured image. The part 
20 thus determined is now referred to as a rendering estimated path. 
In this manner, the rendering device accordingly renders on the 
captured image the rendering estimated path, which ends right 
before the obstacle. 

[0005] The above conventional drive assistant devices carry 
25 two problems as follows. First, the estimated path is fixedly 




determined in color for display. Thus, even if the color is 
similar in tone to a predominant color of the display image, the 
color is unchangeable. Here, the predominant color is mainly 
determined by the road, for example, whether paved or not with 
5 asphalt. If this is the case, the driver finds it difficult to 
instantaneously locate the estimated path on the display image. 
[0006] Second, the estimated path rendered in the display 
image is represented simply by lines, failing to help the driver 
instantaneously perceive how far he/she can move the vehicle. 
?S 10 More specifically, as shown in FIG. 21, a vehicle Vusr carrying 
q the conventional drive assistant device is moving toward an 

yi obstacle Vbst. In this case, the vehicle Vusr first collides a 

p corner point Pcnr of the obstacle Vbst , not intersection points 

fy Pcrg of an estimated path Pp and the surface of the obstacle Vbst. 

□ 15 It means that the farthest point possible for the vehicle Vusr 
to move is the corner point Pcnr of the obstacle Vbst. As such, 
even if the estimated path is so rendered as to end immediately 
before the object, the second problem remains yet unsolved. 

20 SUMMARY OF THE INVENTION 

[0007] Therefore, an object of the present invention is to 
provide a rendering device, a display image being generated 
thereby shows an estimated path in an eye-catching manner for the 
driver to easily locate. 

25 Another object of the present invention is to provide 



a rendering device, a display image generated thereby being 
indicative and helpful for the driver to know how far he/she can 
move the vehicle . 

[0008] The present invention has the following features to 
5 attain the objects above. 

[0009] A first aspect of the present invention is directed to 
a rendering device for generating a display image of around a 
vehicle for drive assistance. The rendering device comprises a 
reception part for receiving a current rudder angle of a steering 

10 wheel of the vehicle from a rudder angle sensor fixed therein; 
a derivation part for deriving an estimated path for the vehicle 
to take based on the rudder angle received by the reception part; 
and an image generation part for generating the display image 
based on a captured image captured by an image capture device fixed 

15 in the vehicle, and the estimated path derived by the derivation 
part. Here, in the display image, the estimated path is overlaid 
on an intermittent basis. 

[0010] A second aspect of the present invention is directed 
to a rendering device for generating a display image of around 

20 a vehicle for drive assistance. The rendering device comprises 
a first reception part for receiving a distance to an obstacle 
located around the vehicle from a measuring sensor placed in the 
vehicle; a first derivation part for deriving -a farthest point 
for the vehicle to move based on the distance received by the first 

25 reception part; a second reception part for receiving a current 




rudder angle of a steering wheel of the vehicle from a rudder angle 
sensor fixed in the vehicle; a second derivation part for deriving 
an estimated path for the vehicle to take based on the rudder angle 
received by the second reception part; and an image generation 
5 part for generating the display image based on a captured image 
captured by an image capture device fixed in the vehicle, the 
farthest point derived by the first derivation part, and the 
estimated path derived by the second derivation part. 
[0011] These and other objects, features, aspects and 
10 advantages of the present invention will become more apparent from 
the following detailed description of the present invention when 
taken in conjunction with the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
15 [0012] FIG. 1 is a block diagram showing the hardware structure 
of a rendering device Urndl according to a first embodiment of 
the present invention; 

FIG. 2 is a diagram showing a display image Sout 
generated by a processor 1 of FIG. 1; 
20 FIG. 3 is a diagram showing a position where an image 

capture device 4 of FIG. 1 is placed; 

FIG. 4 is a diagram showing a captured image ^pi- 
captured by the image capture device 4 of FIG. 1; 

FIG. 5 is a flowchart showing the processing procedure 
2 5 of the processor 1 of FIG. 1; 



[0013] FIG. 6 is a diagram showing a left -side trajectory Ppl 
and a right-side trajectory Pp2 derived in step S6 in FIG. 5; 

FIG. 7 is a diagram showing overlaying position data 
Dsp generated in step S7 in FIG. 5; 
5 FIG. 8 is a diagram showing the display image Sout 

generated in step S8 in FIG. 5; 

FIG. 9 is a diagram showing the display image Sout 
generated in step S15 in FIG. 5; 

FIG. 10 is a block diagram showing the hardware 
10 structure of a rendering device Umd2 according to a second 
embodiment of the present invention; 

[0014] FIG. 11 is a diagram showing a display image Sout 
p generated by a processor 21 of FIG. 10; 

fy FIG. 12 is a flowchart showing the processing procedure 

Q 15 of the processor 21 of FIG. 10; 

FIG. 13 is a block diagram showing the hardware 
structure of a rendering device Urnd3 according to a third 
embodiment of the present invention ; 

FIG. 14 is a diagram showing a display image Sout 
20 generated by a processor 41 of FIG. 13; 

FIGS. 15A and 15B are diagrams showing placement 
positions of active sensors 441 to 444 of FIG. 13; 
[0015] FIG. 16 is a flowchart showing the processing procedure 
of the processor 41 of FIG. 13; 
25 FIG. 17 is a diagram for demonstrating the process in 



step S43 in FIG. 16; 

FIG. 18 is a diagram for demonstrating the process in 
step S44 in FIG. 16; 

FIG. 19 is a detailed diagram showing an estimated 
region Rpt generated in step S410 in FIG. 16; 

FIG. 20 is a diagram showing a display image displayed 
by a conventional drive assistant device; and 

FIG. 21 is a diagram for explaining problems unsolvable 
by the conventional drive assistant device. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0016] FIG. 1 is a block diagram showing the hardware structure 
of a rendering device Urndl according to a first embodiment of 
the present invention. In FIG. 1, the rendering device Urndl 
includes a processor 1, program memory 2, and a working area 3. 
The program memory 2 is typified by ROM (Read Only Memory) , and 
stores a program PGa for defining the processing procedure in the 
processor 1. By following the program PGa , the processor 1 
generates such display image Sout as shown in FIG. 2. The display 
image Sout shows a path Pp estimated for a vehicle Vusr (see FIG. 
3 ) to take in the course of time . The estimated path Pp is composed 
of a left -side trajectory Ppl and a right-side trajectory Pp2 
indicated by, respectively, indicators Sindl and Sind2 '. Here, 
the left-side trajectory Ppl is for a left-rear wheel of the 
vehicle Vusr, while the right-side trajectory Pp2 by a right- 



rear wheel . Further , the indicators Sindl and Sind2 are both 
objects in a predetermined shape (e.g., circle, rectangle) 
previously stored in the program memory 2. 

[0017] The working area 3 is typified by RAM (Random Access 
5 Memory) , and used when the processor 1 executes the program PGa. 
The rendering device Urndl in the above structure is typically 
incorporated in a drive assistant device Uastl . The drive 
assistant device Uastl is mounted in the vehicle Vusjt, and 

Q 

includes at least one image capture device 4 , a rudder angle sensor 

£8 

m 10 5 , and a display device 6 together with the rendering device Urndl . 
O [0018] As shown in FIG. 3, the image capture device 4 is 

HI embedded in the rear-end of the vehicle Vusjt, and captures an image 

O covering an area rear of the vehicle Vusjt. The resulting image 

FU is a captured image Sept as shown in FIG. 4. The rudder angle 

J= 5 15 sensor 5 detects a rudder angle 9 of the steering wheel of the 
vehicle Vusr, and transmits it to the processor 1. The rudder 
angle 6 here indicates at what angle the steering wheel is turned 
with respect to the initial position. The steering wheel is 
considered in the initial position when not turned, that is, when 
20 the vehicle Vusjt is in the straight-ahead position. The display 
device 6 is typically a liquid crystal display. 
[0019] Described next is the operation of such drive assistant 
device Uastl. When the driver wants assistance by the drive 
assistant device Uastl, the processor 1 starts executing the 
25 program PGa. 

8 



[0020] Refer now to a flowchart of FIG. 5 for the processing 
procedure in the processor 1 written in the program PGa. In FIG. 
5, the processor 1 first generates an image capture instruction 
Icpt , and transmits it to the image capture device 4 (step SI). 
5 Here, as shown in FIG. 5, the procedure returns to step SI after 
step S10 is through, and the processor 1 generates another image 
capture instruction Icpt. The program PGa is so written that a 
time interval between those two image capture instructions Icpt 
JJ is substantially a tl second. Here, the value of tl is so selected 

gg 10 as to allow the display device 6 to display the display image Sout 

m 

p for 30 frames per second. Herein, the image capture instruction 

u) Icpt is a signal instructing the image capture device 4 for image 

□ capturing. The image capture device 4 responsively captures such 

RJ captured image Sept as shown in FIG. 4, and stores it in frame 

p 15 memory (not shown) reserved in the working area 3 (step S2). 

[0021] The processor 1 then watches a deriving timing Tl (step 
S3) . This deriving timing Tl is previously written in the program 
PGa. and allows the processor 1 to derive the left- and right-side 
trajectories Ppl and Pp2 once every t2 second. The value of t2 
20 is selected larger than that of tl (e.g., 0.1 second) since a change 
on a time base in the rudder angle 9 is small. 

[0022] In the deriving timing Tl , the processor 1 generates 
a detection instruction Idtc , and transmits it to the rudder angle 
sensor 5 (step S4 ) . The detection instruction Idtc is a signal 
25 instructing the rudder angle sensor 5 to detect the rudder angle 



9 . The rudder angle sensor 5 responsively detects the rudder 
angle 9, and stores it in the working area 3 (step S5). 
[0023] Based on thus detected rudder angle 9 , the processor 
1 derives the left- and right-side trajectories Ppl and Pp2 (step 
S6). More specifically, derived by the processor 1 here are 
equations respectively for the left- and right-side trajectories 
Ppl and Pp2 under the Ackermann ' s model . Here , in the strict sense , 
the left- and right-side trajectories Ppl and Pp2 are defined as 
being trajectories traced by left- and right -rear wheels of the 
vehicle tor on condition that the driver keeps the steering wheel 
at the currently derived rudder angle 9 . The left -side 
trajectory Ppl calculated by such equation becomes an arc in a 
predetermined length. In more detail, the arc is a segment of 
a circle traceable by the vehicle Vusr around a circling center. 
The radius of the circle is equal to a distance from the circling 
center to a point having a rotation center of the left -rear wheel 
projected onto the road surface. The equation for the right- 
side trajectory Pp2 is similar except that the arc is traced by 
the right -rear wheel , on its rotation center, of the vehicle Vusr. 
[0024] Then, the processor 1 generates overlaying position 
data Dsp indicating where to overlay the two indicators Sindl and 
Sind2 , and stores the data in the working area 3 (step S7). As 
an example, if derived in step S6 are such left- and right-side 
trajectories Ppl and Pp2 as shown in FIG. 6, the processor 1 
calculates two points aO and bO being closest to the vehicle Vusr 



(not shown) on those trajectories Ppl and Pp2 , respectively. The 
processor 1 then calculates a point al being away by a 
predetermined distance Ad from the point aO on the left -side 
trajectory Ppl , and a point bl being away also by Ad from the 
point bO on the right-side trajectory Pp2 . The processor 1 
repeats the same processing until i (where 1 is a natural number 
being 2 or larger) sets of coordinates such as {aO, bO) , {al , 
bl) , (a(i-l) , b{l-l)) are calculated. The sets of 

coordinates are numbered starting from the one closest to the 
vehicle Vusr* Accordingly, as shown in FIG. 7, stored in the 
working area 3 is the overlaying position data Dsp including those 
numbered sets of coordinates. 

[0025] Based on the overlaying position data Dsp and the 
aforementioned captured image Sept, the processor 1 then 
generates a frame of the display image Sout on the frame memory 
(step S8). Here, as already described by referring to FIG. 2, 
the display image Sout is the one having the indicators Slndl and 
S±nd2 overlaid on the captured image Sept. In step S8, more in 
detail, the processor 1 first selects, from the overlaying 
position data Dsp generated in step S7 , a set of coordinates which 
is not yet selected and the smallest in number. In this example, 
since no set has yet been selected, selected now is the set of 
(aO, bO) . The processor 1 then overlays the indicators Slndl and 
Slnd2 onto the points aO and bO in the captured image Sept on the 
frame memory. After this overlaying process, such display image 



Sout as shown in FIG. 8 is generated for one frame on the frame 
memory . 

[0026] The processor 1 then transfers the display image Sout 

on the frame memory to the display device 6 for display thereon 
( step S9 ) . In the current display image Sout on the display device 
6 , the indicator Sindl is overlaid on the point aO on the left -side 
trajectory Ppl . and the indicator Sind2 on the point bO on the 
right-side trajectory Pp2 . 

[0027] Then, the processor 1 determines whether now is the time 
to end the processing of FIG. 5 (step S10). If determined not 
yet, the procedure returns to step SI for generating another 
display image Sout. By the time when steps SI and S2 are through, 
another captured image Sept is newly stored on the frame memory. 
Then in step S3, if determining that the timing Tl has not come 
yet, the processor 1 then watches a timing T2 to change the 
overlaying positions of the indicators Sindl and Sind2 ( step Sll ) . 
Here, the changing timing T2 is previously written in the program 
PGa , and allows the processor 1 to change the overlaying positions 
of the indicators Sindl and Sind2 once every t3 second. If the 
value of T3 is set too small, the indicator Sindl moves too fast 
from the point aO to al for the driver to follow with her/his eyes 
on the display device 6. With consideration therefor, the value 
of t3 is selected larger than that of tl (e.g., 0.05 second). 
[0028] If the processor 1 determines that the timing T2 has 
not come yet, the processor 1 generates a frame of the display 



image Sout on the frame memory (step S12) . This is based on the 
captured image Sept stored in step S2 and the set of coordinates 
currently selected in the overlaying position data Dsp (in this 
example, the set of {aO , bO) ) . As such, the resulting display 
5 image Sout is also the one having the indicators Sindl and Sind2 
overlaid on the points aO and bO on the captured image Sept . Then, 
the processor 1 transfers thus generated display image Sout on 
the frame memory to the display device 6 for display thereon ( step 
S S13). 

ff} 10 [0029] Next, in step S10, if the processor 1 determines that 

m 

Q now is not the time to end the processing of FIG. 5, the procedure 

W\ returns to step SI. By the time when steps SI and S2 are through, 

O another captured image Sept is newly stored on the frame memory. 

01 

Then in step S3, if the processor 1 determines that the timing 
~ 15 Tl has not come yet, and in step Sll, if determines that the timing 
T2 is now right, the procedure goes to step S14. Then, the 
processor 1 selects, from the overlaying position data Dsp on the 
working area 3 , a set of coordinates which is not yet selected 
and the smallest in number (step S14). Since the set selected 

20 last is {aO, bO) , selected this time is the set {al . bl) . 

[0030] Next, the processor 1 generates a new frame of the 
display image Sout on the frame memory based on the captured image 
Sept and the set of coordinate (in this example, the set of {al , 
bl) ) currently selected in the overlaying position data Dsp (step 

25 S15). As such, as shown in FIG. 9, the resulting display image 

13 



Sout is the one having the indicators Sindl and Sind2 overlaid 
on the points al and bl on the captured image Sept. Then, the 
processor 1 transfers thus generated display image Sout on the 
frame memory to the display device 6 for display thereon (step 
5 S16). 

[0031] Such steps SI to S16 are repeated until the 
determination in step S10 becomes Yes to end the processing of 
FIG. 5. In this manner, the overlaying positions of the 
~S indicators Sindl and Sind2 change, in increments of the distance 

fg 10 Ad, from the points aO and bO to a{ i-1) and b(i-l) , respectively. 

U1 

P Thus , the indicators Sindl and Sind2 are displayed as if moving 

m in the same heading direction as the vehicle PT/^r along the left- 

Q and right-side trajectories Ppl and Pp2 '. What is good, as the 

m 

Til those indicators Sindl and Sind2 are displayed on an intermittent 

Q 15 basis, the left- and right-side trajectories Ppl and Pp2 are also 
displayed on an intermittent basis on the display device 6. 
Accordingly, the left- and right-side trajectories Ppl and Pp2 
become more noticeable and emphasized to a further degree. With 
such indicators Sindl and Sind2 1 the driver can instantaneously 
20 locate the trajectories Ppl and Pp2 in the display image Sout. 
[0032] Further, every time the rudder angle 0 comes from the 
rudder angle sensor 5 according to the deriving timing Tl , the 
processor 1 derives the left- and right- trajectories Ppl and Pp2 
based on the current rudder angle 6 . In this manner, the 
25 trajectories Ppl and Pp2 displayed on the display device 6 become 

14 



always responsive to the driver's steering. 

[0033] Note that, in the first embodiment, the changing timing 
T2 may be variable. For example, in the case that the overlaying 
positions of the indicators Sindl and Sind2 are relatively close 
to the vehicle Vusr, the program PGa may be so written that the 
changing timing T2 comes earlier. If so, the left- and right-side 
trajectories Ppl and Pp2 become easier to notice.' 
[0034] Further, in the first embodiment, the distance Ad 
between two successive points of aj and a(J+l) is constant on the 
lef t- side trajectory Ppl . Here, the value J is a positive integer 
between 0 and (1-1) . The distance Ad may not necessarily be 
constant . For example, in the case that the point aj is relatively 
close to the vehicle Vusr, the program PGa may be so written that 
the distance Ad is set. relatively small to cause the processor 
1 to select the point a(J+l). Conversely, the program PGa may 
be so written that the distance Ad is set relatively large to 
cause the processor 1 to select the point a(J+l) . In both cases, 
the left- and right-side trajectories Ppl and Pp2 become 
conspicuous to a further degree. 

[0035] FIG. 10 is a block diagram showing the hardware 
structure of a rendering device Urnd2 according to a second 
embodiment of the present invention. In FIG. 10, the rendering 
device Urnd2 includes a processor 21, program memory 22, and a 
working area 23. The program memory 2 2 is typified by ROM (Read 
Only Memory) , and stores a program PGb for defining the processing 



procedure in the processor 21. By following the program PGb , the 
processor 21 generates such display image Sout as shown in FIG. 
11. The display image Sout shows an estimated path Pp of the 
vehicle Vusr (see FIG. 3) to be traced by a left -rear wheel of 
5 the vehicle Vusr. The estimated path Pp is displayed only during 
a display time period Pdt . which will be later described. 
[0036] The working area 3 is typified by RAM (Random Access 
Memory) , and used when the processor 21 executes the program PGb* 

Q 

The rendering device Uirnd2 in the above structure is typically 

W 

rg 10 incorporated in a drive assistant device Uast2 . Here, as to the 
p drive assistant device Uast2 , the only structural difference from 

Ln the drive assistant device Uastl is including the rendering device 

O Urnd2 instead of Urndl . Thus, any component appeared in FIG. 1 

pi 

fU is under the same reference numeral in FIG. 10, and not described 

D 15 again. 

[0037] Described next is the operation of such drive assistant 
device Uast2 . When the driver wants assistance by the drive 
assistant device Uast2 , the processor 21 starts executing the 
program PGb in the program memory 22. 
20 [0038] Refer now to a flowchart of FIG. 12 for the processing 
procedure in the processor 21 written in the program PGb. 
Compared with FIG. 5, the flowchart of FIG. 12 includes the same 
steps, and thus those are under the same step numbers and not 
described again. 

25 [0039] First, by going through steps SI to S6, the processor 

16 



21 derives an equation for the estimated path Pp. The procedure 
then goes to step S21, and the processor 21 generates the display 
image Sout based on the captured image Sept stored in step S2 and 
the estimated path Pp derived in step S6 . More specifically, the 
processor 21 renders thus derived estimated path Pp in its 
entirety on the display image Sout, and the resulting display 
image Sout looks as shown in FIG. 11. 

[0040] The procedure then goes to step S9, and the processor 
21 transfers the display image Sout currently on the frame memory 
to the display device 6 for display thereon. Then, the processor 
21 determines whether now is the time to end the processing of 
FIG. 12 (step S10), and if not yet, the procedure returns to step 
SI for generating another display image Sout on the frame memory. 
By the time when steps SI and S2 are through, another captured 
image Sept is newly stored on the frame memory. Then in step S3, 
if determines that the timing Tl has not come yet, the processor 
1 then determines whether now is in the display time period Pdt 
for the estimated path Pp (step S22). Here, the display time 
period is previously written in the program PGb, and comes every 
t4 second in this embodiment. It means that the estimated path 
Pp appears on and disappears from the display with a time lapse 
of t4 second. Note that, if the value of t4 is set too small, 
the appearance and disappearance of the estimated path Pp will 
be too swift for the driver to notice. With consideration 
therefor, the value of t4 is selected larger than that of tl (e.g. , 




0 . 1 second) . 

[0041] If the processor 21 determines that now is in the display 
time period Pdt , the procedure goes to step S21. The processor 
21 then generates, on the frame memory, the display image Sout 
5 including the estimated path Pp (see FIG. 11) . The procedure then 
goes to step S9, and the processor 21 transfers the current display 
image Sout on the frame memory to the display device 6 for display 
thereon. Then, the processor 21 determines whether now is the 
2 time to end the processing of FIG. 12 (step S10) , and if not yet, 

2 10 the procedure returns to step SI for generating another display 
p image Sout. In step S3, if the processor 21 determines that the 

yn deriving timing Tl has not come yet, and in step S22, if determines 

Q that now is not in the display time period Pdt , the procedure goes 

En 

fU to step S23. In step 23, the processor 21 transfers, to the 

pa 

P 15 display device 6 for display, the captured image Sept stored in 
step S2 (see FIG. 4) as the display image Sout without any change 
( step S23) . 

[0042] Such steps SI to S23 are repeated until the 
determination in step S10 becomes Yes to end the processing of 

20 FIG. 12. In this manner, the estimated path Pp is displayed only 
during the display time period Pdt . The estimated path Pp appears 
on and disappears from the display on an intermittent basis. 
Accordingly, the estimated path Pp becomes noticeable, and the 
driver finds it easy to locate the estimated path Pp in the display 

25 image Sout. 
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[0043] FIG. 13 is a block diagram showing the hardware 
structure of a rendering device Umd3 according to a third 
embodiment of the present invention. In FIG. 13, the rendering 
device Urnd3 includes a processor 41, program memory 42, and a 
working area 43. The program memory 42 is typified by ROM (Read 
Only Memory) , and stores a program PGc for defining the processing 
procedure in the processor 41. By following the program PGc. the 
processor 41 generates such display image Sout as shown in FIG. 
14. The display image Sout shows an estimated region Rpt on a 
road surface Frd for the vehicle Vusr (see FIG. 3) to move. 
Specifically, the estimated region Rpt is defined by the left- 
and right-side trajectories Ppl and Pp2 described in the first 
embodiment, and a line segment Llmt passing through a no-go point 
Plmt . Here, the no-go point Plmt is a point indicating the 
farthest limit for the vehicle Vusr to move, and if the vehicle 
Vusr keeps moving, it might collide the obstacle Vbst first. 
[0044] The working area 43 is typified by RAM (Random Access 
Memory) , and used when the processor 41 executes the program PGc. 
The rendering device Urnd3 in the above structure is typically 
incorporated in a drive assistant device Uast3. Here, as to the 
drive assistant device Uast3 , the structural difference from the 
drive assistant device Uastl is including the rendering device 
Umd3 instead of Urndl , and further including 4 active sensors 
441 to 444, which is exemplified for a measuring sensor in Claims. 
These are the only structural differences , and thus any component 



appeared in FIG. 1 is under the same reference numeral in FIG. 
13, and not described again. 

[0045] As shown in FIG. 15A, the active sensors 441 to 444 are 
embedded in the rear-end of the vehicle Vusjt, preferably, in a 
5 lateral direction. The active sensors 441 to 444 arranged as such 
emit ultrasonic waves or radio waves toward the area rear of the 
vehicle Vusjt, and monitor reflected waves. Thereby, as shown in 
FIG. 15B, distances dl to d4 to an obstacle Vbst located closest 

i i 

40 behind the vehicle Vusr are detected by the active sensors 441 

S 10 to 444. 

q [0046] Described next is the operation of such drive assistant 

q 

yi device Uast3. When the driver wants assistance by the drive 

Q assistant device Uast3 , the processor 41 starts executing the 

Ty program PGc in the program memory 42. 

15 [0047] Refer now to a flowchart of FIG. 16 for the processing 
procedure in the processor 41 written in the program PGc . In FIG. 
16, the processor 41 first generates a distance measuring 
instruction Imsr. and transmits it to all of the active sensors 
441 to 444 (step S41). Here, the distance measuring instruction 
20 Imsr is a signal to instruct all of the active sensors 441 to 444 
to detect the distances dl to d4 , and transmit those to the 
processor 41. The active sensors 441 to 444 each responsively 
perform such detection, and store the resultant distances dl to 
d4 to the working area 43 (step S42). 
25 [0048] Next, based on thus detected distances tfi to d4 , the 

20 



processor 41 calculates coordinates (xl , yl) to (x4 . y4) of four 
points PI to P4 on the surface of the object Vbst (step S43). 
Referring to FIG. 17, the process in step S43 is described in detail . 
FIG. 17 shows the vehicle Vusjt, the obstacle Vbst, and a two- 
5 dimensional (2D) coordinate system. In the 2D coordinate system, 
the Y-axis connects a rotation center of a left -rear wheel Wrl 
and that of a right-rear wheel Wir2 . With respect to the Y-axis, 
the X-axis is a perpendicular bisector parallel to a horizontal 
plane. As described above, the active sensors 441 to 444 are 

So 10 securely placed in the vehicle Vusjt. Therefore, positions Al to 

LP 

O A4 of the active sensors 441 to 444 from which the ultrasonic waves , 

Ul for example, are emitted can be all defined by coordinates (xal , 

Q yal) to (xa4 , ya4) known in the 2D coordinate system. Also, angles 

pi 

: y </> 1 to <t> 4 at which the active sensors 441 to 444 emit the ultrasonic 

— 15 waves are known. In this embodiment, the angles <t> 1 to 4> 4 are 
formed by the X-axis and the emitted waves , and FIG. 17 exemplar ily 
shows only the angle <t> 1 . As such, the above coordinates (xl , 
yl) is equal to (dl • cos<t> 1+xal , dl ' sln<t> 1+yal) , and those (x2 , 
y2) to (x4 to y4) are equal to (d2 • cos<£> 2+xa2 , d2 * sln<t> 2+ya2) 
20 to (d4 * cos<t>4+xa4, d4 • sJ.n<t> 4+ya4) , respectively. 

[0049] Then, based on thus calculated four points ^ to P4 , 
the processor 41 calculates coordinates (xlmt , ylmt) of the corner 
point Pcnr of the obstacle Vbst as one example of the no-go point 
i>/7ff£- (step S44). By referring to FIG. 18, the process in FIG. 
25 44 is now described in detail. The processor 41 first performs 
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Hough transform with respect to the points PI to P4 so that curves 
CI to C4 are derived in the Hough space which is defined by the 
p -axis and ^ -axis. Here, the curves CI to C4 are expressed as 
the following equations (1) to (4), respectively. 



p = 


xl < 


' cos 9 +yl ' 


' sin 


e 


...(1) 


p = 


x2 " 


' cos 9 +y2 ' 


' sin 
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.(2) 


p = 


x3 ' 


' cos 9 +y3 ' 


• sin 


9 


. . .(3) 


p = 


x4 


' cos 9 +y4 1 


• sin 


9 


. . .(4) 



[0050] According to the above equations (1) and (2), the 
processor 41 calculates coordinates (PI, #1) of an intersection 
point Pel of the curves CI and C2 in the Hough space, and according 
to the equations (2) to (4), calculates coordinates (P2, 9 2) 
of an intersection point Pc2 of the curves C2 to C4 in the Hough 
space. From the intersection point Pel , the processor 41 then 
derives an equation for a straight line PI P2 . Here, the line 
PI P2 is expressed by the following equation (5) on the 2D 
coordinate system. Similarly, a line P2 P4 is expressed by an 
equation ( 6 ) . 

y = (-cos 9l • x+pl)/sin0 1 ...(5) 

y = (-cos 9 2 • x+p2)/sin0 2 --.(6) 

From those equations (5) and (6), the processor 41 
calculates coordinates of an intersection point of the line PI 
P2 and the line P2 P3 , and the resulting coordinates are determined 
as the above-mentioned coordinates {xlmt , ylmt) . 
[0051] By similarly going through steps S4 and S5 of FIG. 5, 



the processor 41 receives the current rudder angle 6 of the 
vehicle Vusr (steps S45 and S46). 

[0052] The processor 41 then calculates, in the 2D coordinate 
system, coordinates {xcnt, ycnt) of a center point Pent (see FIG. 
19) of the circle traceable by the vehicle ^rwhen rotated (step 
S47). The processor 41 also derives equations for circles Crl 
and Cr2 , which are traced respectively by the left- and right -rear 
wheels Wrl and Wr2 , on each rotation center, of the vehicle Vusr 
when rotated around the center point Pent (step S48) . Here, since 
the coordinates (xcnt. ycnt) , and the equations for the circles 
Crl and Cr2 are easily calculated under the well-known Ackermann ' s 
model, steps S4 7 and S48 are not described in detail. Further, 
the circles Crl and Cr2 include the left- and right-side 
trajectories Ppl and Pp2 described in the first embodiment. 
[0053] The processor 41 then derives an equation for a straight 
line Llmt , which passes through the coordinates (xcnr t yenr) 
calculated in step S44 , and the coordinates (xcnt, ycnt) 
calculated in step S4 7 (step S49) . Herein, the straight line Llmt 
specifies the farthest limit for the vehicle Vusr to move without 
colliding the obstacle Vbst. 

[0054] The processor 41 next generates the estimated region 
Rpt . which is a region enclosed by the circles Crl and Cr2 
calculated in step S48, the straight line Llmt calculated in step 
S49, and a line segment Lrl2 (step S410) . Here, the line segment 
Lrl2 is the one connecting the rotation centers of the left- and 



right -rear wheels Wrl and Wr2 . 

[0055] By similarly going through steps SI and S2 of FIG. 5, 
the processor 41 receives the captured image Sept from the image 
capture device 4 (steps S411, S412) . Based on the captured image 
Sept and the estimated region Rpt generated in step S410, the 
processor 41 then generates the display image Sout on the frame 
memory. More specifically, the processor 41 deforms the 
estimated region Rpt to the one viewed from the image capture 
device 4, and renders that on the captured image Sept, The 
resulting display image Sout looks as the one shown in FIG. 14. 
The processor 41 then transfers the display image Sout on the frame 
memory to the display device 6 for display thereon (step S414). 
Such steps S41 to S414 are repeated until the determination 
becomes Yes in step S415 to end the processing of FIG. 16. As 
such, as the estimated region Rpt extends to the no-go point Plmt , 
the driver can instantaneously know the farthest limit to move 
the vehicle Vusr. 

[0056] In the above first to third embodiments, the image 
capture device 4 is embedded in the rear-end of the vehicle Vusr. 
This is not restrictive, and in the front -end of the vehicle Vusr 
will also do. Further, the number of image capture devices 4 is 
not limited to one, and may be more depending on the design 
requirements of the drive assistant devices Uastl to Uast3. 
[0057] Still further, in the above embodiments, the captured 
image Sept is the one on which the left- and right-side 
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trajectories Ppl and Pp2 . the estimated path Pp , and the estimated 
region Rpt are rendered. Here, the captured image Sept may be 
subjected to some image processing by the processors 1, 21, and 
41 before having those rendered thereon. Such image processing 
is typified by processing of generating an image of around the 
vehicle Vusr viewed from a virtual viewpoint set high up the 
vehicle Vusr* 

[0058] Still further, in the above first to third embodiments, 
the captured image Sept is stored in the frame memory in response 
to the image capture instruction lept transmitted from the 
processors 1, 21, and 41 to the image capture device 4. This is 
not restrictive, and the captured image Sept is voluntarily 
generated by the image capture device 4 , and then stored in the 
frame memory. Similarly, the rudder angle 6 may be detected 
voluntarily by the rudder angle sensor 5 without responding to 
the detection instruction Idct coming from the processors 1, 21, 
and 41. 

[0059] Still further, in the above third embodiment, four 
active sensors 441 to 444 are placed in the vehicle Vusr. The 
number thereof is not restrictive, and may be one or more. Here, 
if only one active sensor is placed, the direction of the lens 
thereof needs to be dynamically changed so that the angle <t> of 
the emitted waves is set wider. 

[0060] Still further, in the above third embodiment, the 
active sensors 441 to 444 aire provided as one example of a measuring 



sensor in Claims for measuring the distances dl to d4 to the 
obstacle Vbst . This is not restrictive, and other type of 
measuring sensor such as a passive sensor may be used. Here, to 
structure such exemplary passive sensor, two image capture 
devices are required to cover the area rear of the vehicle Vusr. 
These image capture devices each pick up an image of the obstacle 
Vbst located behind the vehicle Vusr. Based on a parallax of the 
obstacle in images, the processor 41 then measures a distance to 
the obstacle Vbst with stereoscopic views (stereoscopic vision) . 
[0061] Still further, in the above embodiments, the programs 
PGa to PGa are stored in the rendering devices Uzrndl to Umd3 , 
respectively. This is not restrictive, and those programs PGa 
to PGa may be distributed in a recording medium typified by CD-ROM, 
or over a communications network such as the Internet. 
[0062] While the invention has been described in detail, the 
foregoing description is in all aspects illustrative and not 
restrictive. It is understood that numerous other modifications 
and variations can be devised without departing from the scope 
of the invention . 



